The present invention relates to an optoelectronic colored image converter having an imaging system which images an object into a two-dimensional CCD-array, the matrix of which being made of light-sensitive sensor elements is provided with a color filter mask for picking up images with at least three colored takes.
Optoelectronic colored image converters having a single CCD-array and color filter mask are commonly known. Compared to colored image converters operating with one CCD-array for each individual colored take, by way of illustration red, green and blue, they have the advantage of having less elaborate construction. The disadvantage, however, is that the number of light-sensitive elements at disposal for a colored take--depending on the design of the colored filter mask--is on the average less by the factor 3 than is the case with optoelectronic: colored image converters working with three CCD-arrays, that is with one per primary color. In addition to the thereby reduced picture resolution, it is also a disadvantage that the colored takes are created by scanning the image at adjacent, thus not congruent image scanning points. This can result, in particular, in the case of fine, periodical image structures in very disturbing color artifacts in the form of color patterns.
From WO 86/05641 and WO 86/05642, optoelectronic colored image converters of another class are known than that described thus far. These are colored image converters having three CCD-arrays in which the image is moved relative to the CCD-array a fraction of the distance between the sensor element (hereinafter referred to as SEL). This shifting takes place in order to raise the picture resolution measured in picture elements (PEL =picture element) over the resolution given by the SEL number.
An object of the present invention is to provide an optoelectronic colored image converter for raising the resolution and for preventing colored scanning artifacts (color aliasing) in a colored image converter having only one CCD-array and color filter matrix, a relative-shifting between the image and the CCD-array.
In accordance with the present invention it was understood that simply transferring the "Sub-Pixel-Relative-Shifting" known from WO 86/05641 or WO 86/05642 is not enough. Namely, in order to prevent color artifacts, shifting by integral multiples of the distance between the sensor elements is required.
By taking several, minimally staggered images (hereinafter referred to as frames) and subsequently superimposing the frames, a more highly resolved picture is yielded. The colored takes of the resulting picture have congruent scanning points in contrast to the colored takes of frames.
For this reason, an object of the present invention is to improve an optoelectronic colored image converter having a CCD-array and a color filter mask so that higher resolution is yielded compared to the number of the CCD-sensor elements (SEL) of the respective color, which meets the requirements of all the color takes at the same scanning points.
This and other objects are met by the present invention which provides an optoelectronic colored image converter having an imaging system, which images an object onto a two-dimensional CCD-array. The matrix of the CCD-array, having light sensitive elements, has a color filter mask for picking up images with at least three color takes. Means are provided for shifting the image relative to the CCD-array between taking consecutive frames in such a manner that the sensor elements of the CCD-array sensitive for the various colors come consecutively to rest on the same point of the image. A memory and control unit congruently composes the color takes of the frames picked up with the shifted CCD-array.
In accordance with the present invention it was understood that in order to raise the resolution of an optoelectronic colored image converter having only one image pick-up before which a color filter mask for taking at least three color takes, thus by way of illustration a red, a green and a blue take, is arranged, it is necessary to first attain by relative shifting between the image and the image pick-up that the sensor elements of the image pick-up sensitive to red, green and blue of the CCD-array come to rest on the same point of the image.
Accordingly, means are provided in an embodiment of the invention, to shift the CCD-array or an element in the beam path before the CCD-array. In this manner, first frames, by way of illustration comprising three color takes each, which have been shifted relative to one another by integral multiples of the SEL distance, are obtained. The number of image points of a frame equals the number of sensor elements of the CCD-array.
A memory and control unit, which intermediately stores the single frames, subsequently congruently composes the color takes of all the frames with the relative shifted CCD-array.
By means of this measure, which is an element of the present invention, due to the relative shifting by integral multiples of the SEL distance the "principle unsharpness" of an optoelectronic colored image converter is obviated, which occurs because the individual color take-pictures are picked up at "different image points". The elimination of this "principle unsharpness" is a prerequisite for sub-SEL shifting, i.e. a relative shifting between the image and the image pick-up by a fraction of the distance between the individual light-sensitive sensor elements. This sub-SEL shifting, in addition, permits raising the resolution continuously at this rate, the smaller the dimensions of the edges of the light-sensitive area of a sensor element relative to the distance between the single elements.
The shifting of the CCD-array has the advantage compared to shifting or tilting an element in the beam path for the CCD-array in that it is technically easier to realize. By way of illustration, piezoelements can be provided, which shift the CCD-array one or two-dimensionally in the plane of the image.
The previously mentioned "sub-SEL-shifting" described can also be utilized to obtain an assimilation of the image resolution when the distances between the sensor elements differ in perpendicular directions to one another.
Furthermore, it is advantageous that an embodiment of the present invention also permits scanning in a hexagonal pattern corresponding to the densest filling of an area.
In addition, the following advantages are attained by the small relative shifting of a CCD area sensor and subsequent composition of the frames provided for in accordance with the present invention.
Very high resolution of the entire picture of, by way of illustration 2000.times.1650 image elements, is obtained for all three color channels with the use of interline transfer color sensors, which in the case of the described embodiments has, by way of example, 250.times.550 SEL for the green channel and 250.times.275 SEL for the red and the blue channels and light-sensitive area elements having dimensions of 6.times.6 .mu.m.
The modulation transfer function of the sensor has fallen to approximately 30% with this resolution, permitting in particular high-resolution scanning of a DIN A4 printed page, a color slide or a strip of film with a picture format of 24.times.18 mm.
Geometric precision is already very high in normal mechanical embodiments of the shifted elements as the shift paths are very short. In the aforementioned example, the size of the resolution cell in a color sensor is 34.times.44 .mu.m, whereas in a corresponding B/W sensor it is 17.times.11 .mu.m. A relative error of the mechanical shifting by, for example 1%, results in a maximum error of 0.44 .mu.m (mechanical) plus 0.1 .mu.m of the sensor in the entire image field. This is small compared to the radial lens distortion of CCTV lenses, which is, by way of illustration, already up to 50 .mu.m at the edge of the picture in a lens with a focal distance of 25 mm.
Light-sensitivity is raised compared to line scanners or drum scanners by orders of magnitude so that the pick-up time is substantially shortened while having the same resolution and the same signal/noise ratio.
Having different shift patterns permits, within limits, choosing freely between high local resolution and high temporal resolution.
Furthermore, it is possible to scan the image not only in a rectangular, but also in an almost hexagonal raster, which is occasionally required, in particular, in medicine and in morphological image processing.
The selected detail of the picture can be monitored with a normal TV monitor in full temporal resolution and thus be sent without difficulty prior to the actual pick-up as the frames are the same size as the entire picture.
As the electric bandwidth of the frames at the output of the sensor corresponds to that of a TV picture, the frames can be first intermediately stored with conventional commercial video recorders and then later composed to an entire high-resolution picture by the computer, permitting using it in a portable electronic color camera having practically the resolution of a miniature camera slide and a memory capacity of more than a thousand single pictures. Magnetic band drop-outs can be easily detected by the computer due to the image-boxing and corrected without very visible losses (the principle of data-boxing or distribution).
Due to the short mechanical paths, no slide guides are needed, only bend guides. For the same reason piezoelectric positioning elements without backlash can be utilized, which can be directly electrically triggered and are mechanically very stable.
As the color mask can be attached directly onto the sensor, so-called color multiplex does not have to be generated like in the case of black-white sensors by the color filters having the size of the image field placed one behind the other in the beam path or by color separation with prisms onto three sensors, but simply occurs by shifting the sensor laterally. Chromatic aberrations or other impairments of the image from the filters are therefore obviated so that aberrations are only caused by the lens.
There is a cost-saving due to the use of mass-produced low resolution CCD area sensors, using the present invention.
There is a high resolution of, for example, 2000.times.1650 image elements, and with a reduction of the light-sensitivity of the area elements even more for each of the three color channels, that can be attained in embodiments of the present invention.
Light-sensitivity is approximately two orders of magnitude larger than that of cameras having a shifting line, which have comparable image resolution.
The precision of the position of the image scanning points is also approximately two orders of magnitude larger than that of such line cameras.
The costs of the colored image converter of the present invention are very low due to the use of only a single mass-produced consumer electronics sensor without any elaborate optics or expensive color filter.
In particular, the invention has some of the following uses.
The present invention can be used in data teletransmission, either B/W or color telefax. Data pick-up can occur in a shorter time and with lower demands on the power of the light source. Due to the use of area sensors, the original no longer needs to be shifted. For this reason, extremely high geometric precision in image pick-up can be guaranteed.
The present invention can also be used in film scanning for digital film reprocessing. In particular, when coloring, compensating for color errors or in the case of data input for "special effects", due to the great number of images to be picked up, short pick-up time is of special interest. Although flying spot scanners can attain the required resolution, they are very expensive due to the elaborate technology with three photomultipliers (several hundred thousand DM). Line cameras are less sensitive and due to the major mechanical shifting cause the image to flicker because of the geometric imprecisions.
The present invention can also be used in high-resolution data input of single images for commercial arts. Nearly all full page color advertising photographs in magazines are digitally improved prior to printing by raising edges, suppressing noise, increasing color saturation and contrast, and retouching.
The present invention can also be used in high-resolution data input for application in photometry. The use of digital image processing in photometry still suffers, in particular, from insufficient availability of geometrically highly precise and high-resolution scanning systems. The image converter of the present invention closes this gap.
The present invention can also be used for Image Scanning for Electronic Filing (Document Filing). With the available resolution, diapositives and dianegatives can be filed with almost no losses, which is of significance for setting up data banks in medicine for example.
The present invention can also be used in an Electronic Diapositive/negative Viewer. With the invented color image converter and a high-resolution monitor, diapositives and with digital matrixing also dianegatives can be viewed in high quality, which will gain in significance with the foreseeable establishment of HDTV standards. By employing two memories, pictures can be changed without interruption.
The present invention is also useful in Videometry. The availability of a high-precision scanning raster, which can be shifted fractions of the distance between the sensor elements, is of importance for measuring methods with image processing. This is, in particular, true for moire measuring with strip grids as the determination of the phase can occur separately for each image point directly by shifting the image point 120.degree..
The present invention can also be used with portable still cameras with slide quality. The availability of digital tape memories (digital audio tape on video tapes) with high recording density makes, due to the low weight and low required performance of a camera, a portable camera in accordance with the present invention conceivable, which permits storing several hundred pictures on a very inexpensive carrier. Simultaneously, this, permits the "videography" of buildings and excavation pieces in archaeology for measuring purposes as recording occurs with extremely high geometric precision.
Other objects, advantages and novel features of the present invention will become apparent from the following detailed description of the invention when considered in conjunction with, the accompanying drawings.